Optimization by ghost image processes in neural networks

Scope and Purpose-Neural networks come in a variety of forms and are "trained" by a variety of strategies. With a few exceptions, these forms and training processes have not produced strongly competitive approaches for optimization problems. when compared to latest methods that have evolved within the optimization field. This paper proposes a different type of neural network conception based on "ghost image processes", The fundamental idea is to use two reinforcing types of mappings, one operating on trial solutions and one operating on idealized problem representations or target structures (called ghost images). This gives a natural basis for integrating the design and training functions, and provides an effective way to handle a variety of optimization problems that were previously not well suited to be treated by neural networks. The ghost image processes are able to incorporate specialized components to exploit problem structures in specific optimization domains. and to integrate classical optimization and search methods as part of this process. Examples show how ghost image processes can take advantage of optimization problems with diverse characteristics, and preliminary computational tests are reported for multidimensional knapsack problems that demonstrate the promise of these processes. Abstract-We identify processes for structuring neural networks by reference to two classes of interacting mapping~ one generating provisional outcomes ("trial solutions") and the other generating idealized representations. which we call ghost images. These mappings create an evolution both of the provisional outcomes and ghost images, which in turn influence a parallel evolution of the mappings themselves. The ghost image models may be conceived as a generalization of the self-organizing neural network models of Kohonen. Alternatively, they may be viewed as a generalization of certain relaxation/restriction procedures of mathematical optimization. Hence indirectly they also generalize aspects of penalty based neural models. such as those proposed by Hopfield and Tank. Both avenues or generalization are "context free". without reliance on specialized theory. such as models of perception or mathematical duality. From a neural network standpoint. the ghost image framework makes it possible to extend previous Kohonen-based optimization approaches to incorporate components beyond a visually oriented frame of reference. This added level of abstraction yields a basis ror solving optimization problems expressed entirely in symbolic ("non-visual") mathematical fonnulations. At the same time it allows penalty function ideas in neural networks to be extended to encompass other concepts springing from a mathematical optimization perspective. including parametric deformations and surrogate contractions. This paper demonstrates …